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SUMMARY  PAGE 


PROBLEM 


To  determine  human  efficiency  for  visual  detection  of  targets 
on  CRT  displays  using  a  two-level  mulitple  channel  time  history 
format . 


FINDINGS 

Human  observers  were  generally  3-5  dB  less  sensitive  than 
optimal  detectors.  Although  observers’  performance  improved  as 
the  number  of  lines  in  the  display  increased,  this  improvement 
was  not  as  great  as  that  achieved  by  an  optimal  detector, 
particularly  for  lower  signal  levels.  Less  than  optimal 
performance  may  be  due  to  observers'  inability  to  focus  on 
individual  bearings  or  that  they  employ  suboptimal  strategies  for 
using  the  information  at  a  bearing. 

APPLICATION 


The  finding  that  observers  are  3-5  dB  Less  sensitive  than 
theoretically  possible  indicates  that  improved  design  of  visual 
displays  could  lead  to  better  detection  of  sonar  targets. 
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ABSTRACT 


Human  observers  were  tested  for  their  ability  to  detect 
targets  on  visual  displays.  The  displays  simulated  the  multiple 
channel  time  history  format  of  sonar  displays,  using  two  levels 
of  intensity  encoding.  A  target  was  presented  on  50%  of  the 
trials  and  appeared  as  a  vertical  line  at  a  fixed  position. 
Observers  indicated  their  judgment  as  to  whether  a  target  was 
present  by  using  a  four-category  rating  scale. 

Receiver-operating  characteristics  (ROCs)  were  generated  from  the 
rating  data  and  compared  to  ROCs  for  an  optimal  detector.  Data 
were  collected  for  two  signal-to-noi se  ratios  and  for  32,  64,  and 
120  lines  of  data.  Results  indicated  that  observers  were  3-5  dB 
less  sensitive  than  an  optimal  detector.  Performance  improved  as 
the  number  of  lines  increased  but  not  to  the  extent  predicted  by 
optimal  integration  of  information  across  lines  of  the  display. 
Human  inefficiency  with  this  display  is  possibly  due  to  the 
inability  to  focus  on  a  single  column  of  data  or  to  the  use  of  a 
suboptimal  decision  rule  for  judging  the  presence  of  the  target. 
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INTRODUCTION 


Several  investigators  have  experimented  with  target 
detectabi L i ty  in  the  multiple-channel  time-history  format  ( MCTH ) . 
These  displays  are  used  to  present  data  from  passive  sonar 
systems  ana  multichannel  spectrum  analyzers.  The  Passive  Broad 
Band  (PBB)  display  is  an  example  of  this  format.  In  the  PBB 
display,  bearing  is  represented  along  the  horizontal  axis,  time 
along  the  vertical  axis,  and  amplitude  of  the  signal  is  encoded 
by  pixel  intensity.  Currently,  these  displays  are  monochromatic, 
and  signal  strength  may  be  mapped  onto  two,  four,  or  eight 
luminance  levels  (aLso  referred  to  as  brightness  Levels). 


Thompson  (1975)  has  demonstrated  that  target  detectabi l 1 ty 
improves  as  the  number  of  luminance  levels  used  to  display 
information  increases.  There  is  also  evidence  that  displaying  J 
information  in  color  as  opposed  to  monochromatic  luminance  levels  § 
may  improve  target  detectability  (Evans  1968).  Although  these  \ 
experiments  have  provided  empirical  evidence  that  operator  f 

f 

performance  varies  with  display  format,  no  comparison  between  | 

T 

actual  human  performance  and  idea!  performance  has  been  made,  nor  1 

I 

have  the  investigators  attempted  to  specify  those  critical  j 

features  m  the  displays  that  lead  to  improved  performance.  I 


Delisle  and  Kroenert  (1983)  nave  proposed  a  machine  based, 
analytical  model  of  performance  detection.  Given  certain 
constraints  placed  upon  the  target,  tins  model  represents  an 
ideal  detector.  The  purpose  of  our  research  is  to  evaluate 
operator  performance  in  terms  of  ideal  per formance. 

Delisle  and  Kroenert' s  modeL  assumes  that  the  display  is 
composed  of  two  brightness  levels,  on  and  off.  The  probability 
that  noise-only  data  will  appear  on  the  screen  is  0.5.  Thus 
noise-only  data  will  appear  as  a  "random  dot"  display.  The 
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probability  that  the  target  data  will  appear  on  the  screen  is 
greater  than  0.5.  How  much  greater  than  0.5  depends  on  the 
target's  strength;  the  stronger  the  target's  signal  the  more 
likely  its  data  will  appear  on  the  screen.  If  we  assume  that  a 
target's  bearing  is  constant,  that  is,  the  target's  horizontal 
position  in  the  display  remains  constant  over  a  specified  number 
of  temporal  updates  of  data,  then  the  ideal  detector  adjudicates 
whether  a  target  is  present,  based  on  the  number  of  lit  cells  (or 
bins)  in  a  specified  column.  The  ideal  detector  sets  a  criterion 
for  detection;  If  j  or  more  cells  are  lit  over  a  specified 
number  of  lines,  then  a  detection  is  made.  If  these  cells  were 
lit  by  noise,  the  judgment  is  called  a  false  alarm;  if  they  were 
lit  by  a  target,  the  judgment  is  called  a  hit.  When  L  lines  of 
data  are  presented,  the  probability  that  j  or  more  cells  will  be 
lit  by  either  the  noise  or  target  can  be  computed  from  the 
binomial  distribution; 


(1) 

Where 


L 

P(j)  =  ^  Ll/(il*(L-i)l)*pi*qL"i 

i  =  j 


is  the  probability  that  at  least  j 
cells  are  lit. 

p:  is  the  probability  of  a  cell  being  lit. 
q:  is  the  probability  of  a  cell  being  off. 
L:  is  the  number  of  lines  in  the  display, 

i:  index  referring  to  detection  criterion. 


As  mentioned  above,  for  the  noise  distribution,  p  =  q  =  0.5.; 
whereas  for  the  signal  distribution,  p  >  0.5,  q  =  1-p. 

for  noise 


By  plotting  the  respective  probabilities  from  Eg.  1 
versus  target  as  j  varies  from  O  to  L,  we  trace  out  the 


ROC  curve 


(Green  and  Swets  1966)  for  the  two  distributions.  Thus  for  a 
given  false  alarm  rate,  the  hit  rate  can  be  calculated,  and  these 
two  rates  correspond  to  the  performance  of  the  ideal  detector. 

The  performance  of  this  detector  is  ideal  in  the  sense  that  for  a 
given  false  alarm  rate,  it  is  impossible  to  obtain  a  higher  hit 
rate.  Figure  1  depicts  the  ROC  curves  for  a  signal  strength  of 
-2  db  and  -5  db  across  32,  64  and  128  lines  of  data.  The 
probabilities  have  been  converted  to  z-scores,  yielding  lines 
whose  slopes  are  all  approximately  equal  to  1. 

The  curves  in  Figure  1  represent  ideal  performance.  That  is, 
for  a  given  number  of  lines  of  data,  associated  with  each  false 
alarm  rate  (the  x  coordinate)  is  the  best  possible  hit  rate,  (the 
y  coordinate).  If  human  performance  is  equivalent  to  ideal 
performance,  then  the  observer's  false  alarm  rate  and  hit  rate, 
taken  as  a  (x,y)  paired  coordinate,  should  plot  onto  the 
respective  ROC  curve.  In  this  manner,  an  observer’s  performance 
may  be  compared  to  ideal  performance.  Furthermore,  by  having  the 
subject  give  a  confidence  judgment,  performance  can  be  compared 
to  ideal  at  a  variety  of  false  alarm  rates  -  in  essence,  tracing 
out  the  ROC  for  the  human  observer  by  having  them  vary  their 
criteria  for  detection. 

One  consequence  of  Delisle  and  Kroenert’s  model  is  that 
performance  should  improve  as  the  number  of  lines  in  the  display 
increases  (see  Figure  1).  That  is,  the  more  data  available  to 
the  ideal  detector,  the  more  accurate  its  judgment  will  be.  The 
current  research  evaluates  human  performance  for  displays  of 
32,64,  and  128  lines  of  data,  in  order  to  determine  the  degree  to 
which  the  human  observer  integrates  information  when  presented 
with  more  lines  of  information. 

Delisle  and  Kroenert ' s  model  assumes  that  the  observer  is 
only  asked  to  judge  whether  or  not  a  target  is  present  at  one 
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bearing.  Thus  in  their  model  there  is  no  uncertainty  as  to  the 
position  of  the  target,  only  whether  or  not  a  target  is  present. 
In  this  sense  the  model  is  strictly  a  detection  model.  In 
practice  however,  with  such  displays,  the  observer  typically  has 
to  judge  not  only  whether  or  not  a  target  is  present  but  also 
must  decide  where  the  target  is  located,  i.e.  at  what  bearing  the 
target  is  coming  in.  Thus,  the  task  generally  entails  both 
detection  and  identification  (identification  in  this  context 
refers  to  correctly  specifing  the  bearing).  In  our  experiments, 
the  subjects  had  only  to  detect  the  presence  of  the  target;  its 
location  was  specified. 

METHOD 


Three  members  of  the  Vision  Department  at  The  Naval  Submarine 
Medical  Research  Laboratory  were  subjects.  Two  were  experienced 
with  waterfall  displays. 


Apparatus  I 

I 

A  Vax  750,  a  Ramtek  9400  graphics  display  generator,  and  | 

I 

Matsushita  standard  phosphor  color  monitor  were  used  to  simulate  % 

k 

the  sonar  display.  The  addr essabi 1 i ty  of  the  monitor  was  1024  by  | 


1280  pixels  (100  pixels  per  inch).  The  C.I.E.  chromaticity 
coordinates  (x,y)  of  the  phosphors  were  0.60,  0.34  for  the  red, 
0.28,  0.59  for  the  green,  and  0.16,0.07  for  the  blue.  The 
display  was  illuminated  by  two  fluorescent  tubes  located  behind 
and  above  the  observer.  The  lamps  were  covered  with  neutral 
density  filters  which  reduced  the  illumination  falling  on  the 
screen  to  0.25  fc,  as  measured  by  a  Gossen  light  meter. 


Display 


The  display  simulated  one  depression-elevation  sector  of  a 
spherical  array  passive  broadband  (SAPBB)  short  term  averaging 
(STA)  display,  with  bearing  represented  along  the  horizontal 
axis,  time  along  the  vertical  axis,  and  amplitude  of  signal 
encode  by  pixel  intensity.  The  display  may  be  conceptualized  as 
a  rectangular  matrix.  The  number  of  columns,  i.e.  bearings,  was 
always  60,  whereas  the  number  of  rows.  i.e.  temporal  updates, 
was  an  experimental  variable.  Displays  of  32,  64,  and  128  lines 
of  data  were  tested.  At  a  viewing  distance  of  2  ft,  these  three 
displays  subtended  visual  angles  of  2.34  x  4.77  deg,  4.47  x  4.77 
deg,  8.93  x  4.77  deg,  respectively.  A  white  cross  hair  was 
placed  directly  above  the  31st  bearing,  at  approximately  the 
middle  of  the  display. 

One  bin  of  information,  which  corresponds  to  one  element  in 
the  array,  was  represented  by  a  3x3  block  of  pixels.  Only  two 
luminance  levels,  "on"  and  "off",  were  used  to  encode  pixel 
intensity.  The  on  state  utilized  the  green  phosphor,  and 
corresponded  to  a  luminance  value  of  .35  fL. 

The  random  energy  in  the  sea  may  be  simulated  by  a  Gaussian 
distribution  with  a  mean  of  zero  and  a  standard  deviation  of  one. 
The  mapping  of  noise  to  luminance  level,  in  this  case  "on"  or 
"off",  entails  the  arbitrary  assignment  of  values  sampled  from 
the  noise  population  to  luminance  level.  In  this  display,  all 
values  less  than  or  equal  to  zero  mapped  onto  a  luminance  level 
of  screen  off,  whereas  any  value  greater  than  zero  mapped  onto 
screen  on.  That  is,  the  average  marking  density  was  0.5.  The 
signal  distribution  is  a  Gaussian  with  variance  equal  to  that  of 
the  noise  (  i.e.  S.D.=1)  However,  the  mean  of  the  signal's 
distribution,  is  shifted  from  that  of  the  noise,  Mn, 

depending  on  the  strength  of  the  signal.  The  mean  of  the  signal 
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distribution  is  given  by  the  equation: 


2) 


Ms  =  +  10 


SNR/10 


Signal  strengths  (SNRs)  of  -2  dD,  and  -5  db  were  tested  u\ 
the  experiment.  These  values  of  SNR  correspond  to  average 
marking  densities  of  74%  and  63%,  respectively. 

Procedure 

In  the  experiment,  one  trial  consisted  of  the  following 
events.  The  display  waterfalled  at,  a  rate  of  approximately  10 
lines/sec  until  an  entire  new  block  of  data  was  presented,  and 
then  stopped.  The  subject  was  instructed  to  judge  whether  a 
target  was  present  at  the  31st  bearing  only.  (This  bearing  was 
marked  by  a  cross  hair.)  Subjects  rated  the  confidence  of  their 
responses  with  the  following  scale:  l  -  indicated  that  they  were 
relatively  certain  that  a  target  was  not  present,  2  -  no  target 
but  some  uncertainty,  3  -  target  present  but  some  uncertainty,  4 
-  relatively  certain  that  a  target  was  present.  Subjects  had  an 
unlimited  amount  of  time  in  which  to  view  the  static  display  and 
make  their  decision.  After  entering  their  response,  a  new  trial 
started . 

The  probability  that  a  target  was  presented  on  any  one  trial 
was  50%.  As  mentioned,  the  subjects  were  tested  on  displays  of 
32,  64,  and  128  lines  of  data.  The  number  of  lines  of  data  in 
the  display  remained  constant  for  100  trials.  Each  subject 
completed  5  blocks  of  100  trials  at  each  of  the  two  signal  levels 
and  for  each  of  the  three  number  of  lines  tested.  Across  blocks 
of  trials  the  order  of  presentation  of  both  the  number  of  lines 
and  signal  strength  was  randomized.  Subjects  knew  whether  they 
were  searching  for  a  weak  or  strong  target  in  each  block  of 
trials . 
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RESULTS 


The  four  response  categories  were  grouped  to  yield  three 
points  on  the  ROC  curve  in  the  following  manner.  First,  a  lax 
criterion  was  generated  by  considering  a  response  of  1  as  no 
target  present,  and  a  response  of  2,  3,  or  4  as  target  present. 

Second,  a  neutral  criterion  was  generated  by  grouping  response  1 
and  2  as  no  target  present,  and  response  3  and  4  as  target 
present.  Lastly,  a  strict  criterion  was  obtained  by  grouping 
responses  1,  2,  and  3  as  a  no  response  and  4  as  a  yes  response. 
The  results  for  the  32,  64  and  128  line  displays  are  plotted 
separately  for  each  subject  in  Figures  2,  3,  and  4. 

Unfortunately,  we  were  unable  to  obtain  a  data  point  generated  by 
the  strict  criterion  across  all  three  number  of  lines  tested  for 
the  -2  db  target  for  one  subject.  This  subject  had  no  false 
alarms  that  were  rated  a  4  at  this  signal  strength. 


As  is  illustrated  in  Figures  2,  3,  and  4,  we  were  able  to 
obtain  coherent  ROC  curves  for  each  subject.  Ideal  performance 
for  a  signal  strength  of  -2,  -5,  and  -8  db  has  been  plotted  for 
reference.  To  further  analyze  the  results,  best  fitting  lines 
were  fit  to  the  data  points.  Tables  1,  2,  and  3  list  the  slopes, 
average  slopes  and  intercepts  of  these  lines.  The  slopes  of 
these  lines  are  all  approximately  equal  to  one  as  the  model 
predicts.  In  fact,  the  data  are  rather  consistent.  In  general, 
the  subjects  performed  approximately  3  db  below  ideal 
performance.  (Subject  LB,  who  was  previously  unfamiliar  with 
waterfall  displays,  is  almost  5  db  below  ideal  performance.) 

That  is,  subject  performance  on  the  -2  db  target  was  what  one 
would  expect  for  a  weaker  target  of  -5  db,  whereas  performance  on 
the  -5  db  target  had  degraded  to  performance  expected  for  a 
signal  strength  of  -8  db.  Tables  4  and  5  demonstrate  this  point. 
For  given  false  alarm  rates  of  10%  (Table  4)  and  5%  (Table  5), 
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Figure  2.  ROC  curves  for  each  subject  when  tested  on  32  lines  of 
data.  The  ROC  curve  for  Ideal  performance  is  given  for  reference 
by  the  solid  lines  for  signal  strengths  of  -2,  -5,  and  -8  db . 


FALSE  ALARM  RATE  (Z-SCORES) 

Figure  3.  ROC  curves  for  each  subject  when  tested  on  64  lines  of 
data.  The  ROC  curve  for  Ideal  performance  is  given  for  reference 
by  the  solid  lines  for  signal  strengths  of  -2,  -5,  and  -B  db. 


FALSE  ALARM  RATE  (Z-SCORES  ) 

Figure  4.  ROC  curves  for  each  subject  when  tested  on  128  li 
of  data.  The  ROC  curve  for  Ideal  performance  is  given  for 
reference  by  the  solid  lines  for  signal  strengths  of  -2,  -5, 
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TABLE  1:  Slopes  of  best  fitting  line  for  each  subject's  ROC. 
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*  Only  two  points  on  the  ROC  were  obtained. 
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TABLE  4:  Hit  rates,  in  percent,  from  a  10?,  false  ala r m  : 

-  -1.28)  extrapolated  from  best  fitting  lines  for  each  s  i :  i  >  j  < 
each  condition. 
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TABLE  5:  Hit  rates,  in  percent,  from  a  5%  false  alarm  rate  (z 
-1.65)  extrapolated  from  best  fitting  lines  for  each  subject  in 
each  condition. 
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each  subject  s  hit  rate,  extrapolated  trora  the  best  itting  Line, 
may  be  compared  to  the  ideal  hit  rate.  For  example,  with  32 
lines,  a  signal  level  of  -2  db,  and  a  false  alarm  rate  of  10% 
(Table  4),  obtained  hit  rates  range  from  36%  to  62%.  This 
percentage  is  much  less  than  the  94%  possible  for  the  ideal 
detector  with  a  signal  level  of  -2  db,  but  corresponds  well  to 
the  56%  for  the  ideal  detector  with  a  signal  level  of  -5  db. 

Table  6  shows  the  equivalent  loss  in  SNR  for  each  condition 
relative  to  ideal  performance.  The  SNR  loses  were  obtained  in 
the  following  manner.  Values  of  d'  (Green  and  Swots,  1966/1974) 
were  estimated  at  a  false  alarm  rate  of  5%  for  the  ideal  detector 
and  the  three  subjects.  Since,  for  the  ideal  detector,  d'  is 
approximately  proportional  to  signal  energy,  SNR  loss  can  be 
computed  as: 


10  log ( d  '  ,  /d  '  .  j 


ideal 


where  d' 


is  the  d'  for  the  observer  and  d' 


is  the  d'  for 
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the  ideal  observer.  The  values  for  the  SNR  loss  range  from  -1.9 
to  -6.1  db.  Table  7  gives  the  same  information  as  Table  6  except 
averaged  across  subjects.  Tables  6  and  7  show  that  as  the  number 
of  lines  increased,  the  discrepancy  between  subject  performance 
and  ideal  performance  increases.  In  fact,  for  the  128  line 
display  the  discrepancy  is  clearly  qreater  than  -3  do  for  all 
subjects . 


d  is  a  measure  of  sensitivity  defined  as  the  difference 
between  the  mean  of  two  Gaussian  distributions  divided  by  their 
common  standard  deviation. 
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Table  6:  d'  and  SNR  loss  for  each  subject  in  each  condition  for 
a  false  alarm  rate  of  5%. 
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TABLE  7:  Average  d'  and  SNR  xoss  at  a  false  alarm  rate 
for  the  three  subjects.  Ideal  d  '  is  given  for  reference 
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Although  human  performance  was  relatively  poor  as  compared  to 
ideal  performance,  in  general  as  the  number  of  lines  increased  in 
the  display,  subjects'  performance  improved.  This  result  is  in 
accordance  with  Delisle  and  Kroenert ' s  model  and  is  clearly 
illustrated  in  Figure  5  where  SML 1 s  data  are  plotted  for  the  32, 
64  and  128  line  display.  However,  as  noted  above,  this 
improvement  is  not  as  large  as  expected,  particularly  for  an  SNR 
of  -5  db  where  only  a  small  improvement  is  observed  as  a  function 
of  the  number  of  lines  (see  Figure  6).  Table  7  also  shows  the 


FALSE  ALARM  RATE  (  Z- SCORES  ) 


Figure  6.  ROC  curves,  sol  id  lines,  for  subject  SML  on  the  -5  d 
target  for  32,  64  and  128  Lines  of  data.  Performance  improves 
only  slightly  as  the  number  of  lines  increase  from  32  to  64;  wi 
128  lines,  the  slope  of  the  ROC  is  less  than  one,  making  it 
difficult  to  ascertain  if  there  has  been  further  improvement. 
Note  that  performance  is  far  below  the  ROC  curve  for  ideal 
performance  at  only  32  lines  of  data  (broken  line). 


One  factor  that  may  be  detrimental  to  performance  is 
uncertainty  (Green  and  Swets  1966/1974).  Despite  the  fact  that 
the  target's  bearing  was  clearly  marked  by  a  cross  hair,  the 
subjects  still  experienced  some  uncertainty  as  to  which  column 
was  the  target  column.  This  is  because  the  random  dot  pattern  of 
the  noise  makes  it  difficult,  if  not  impossible,  to  clearly 
discern  a  single  column  of  the  display  from  top  to  bottom.  Thus, 
the  subject  may  be  looking  across  several  columns  in  the 
immediate  vicinity  of  the  target  column.  This  would  have  a 
detrimental  effect  on  performance,  because  the  probability  that 
one  of  several  noise-only  bearings  may  appear  to  contain  a  signal 
is  greater  than  the  probability  that  the  single  target  bearing 
will  appear  to  contain  a  signal  when  noise  is  presented  at  that 
bearing.  Thus  the  subject  is  more  likely  to  confuse  noise  for 
signal.  The  extent  to  which  uncertainty  is  degrading  performance 
must  be  evaluated  directly  in  a  separate  experiment. 

Another  factor  which  may  be  detrimental  to  performance  is  the 
reliance  on  the  "clustering"  of  lit  pixels  in  order  to  detect 
targets.  It  is  apparent  that  a  string  of  lit  pixels  clustered 
together  stands  out  from  the  background  as  a  short  line  and 
appears  to  be  a  target.  It  is  generally  true  that  targets  will 
exhibit  more  clustering  than  noise;  however,  given  a  fixed 
marking  density,  i.e.  a  given  number  of  lit  pixels  over  a  given 
number  of  lines,  the  probability  that  these  data  are  from  a 
signal  of  a  particular  strength  is  independent  of  the  clustering 
of  the  Lit  pixels  within  that  string.  Thus,  if  we  have  two  data 
strings  with  equal  marking  density,  as  illustrated  in  Figure  7, 
subjects  should,  ideally,  treat  these  strings  as  equally  likely 
to  contain  a  target.  However,  they  may  be  more  apt  to  claim  that 
string  A  is  a  target  because  the  clustering  of  lit  pixels  is 
greater  in  string  A  than  B.  They  are  thus  adopting  a  suboptimal 
strategy  and  their  performance  departs  from  ideal  performance. 


Figure  7:  Illustration  ot  the  grouping  factor.  The  marking 

densities  of  strings  A  and  B  are  equal;  however,  the  clustering 
factor  is  greater  in  string  A  than  in  string  B. 


Trie  extent  to  which  this  clustennq  factor  lias  an  adv->rs<'  '-1 
on  target  detection  must  also  be  analyzed  by  obtaining  ROC  'urve 
which  assume  different  degrees  of  cluster! pu. 
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